System and method for mobile device active callback prioritization

ABSTRACT

A system and methods for mobile device active callback prioritization, utilizing an enhanced callback prioritization engine operating on a user&#39;s mobile device for integration through the operating system and software applications operating on the device, wherein the enhanced callback prioritization engine receives intercepted data or voice messages sent to the mobile device, retrieves and aggregates data related to the assigned messages, inputs the assigned data message and aggregate data into one or more machine learning algorithms wherein the algorithms may analyze the input data, the results of the analysis may be used to compute a priority score for the assigned data message, and generates a callback list from the computed prioritization score. The priority score is in part based on 3rd party application data related to the data or voice messages providing context to the machine learning algorithms.

CROSS-REFERENCE TO RELATED APPLICATIONS

Priority is claimed in the application data sheet to the followingpatents or patent applications, the entire written description of eachof which is expressly incorporated herein by reference in its entirety:

-   -   Ser. No. 17/572,405    -   Ser. No. 17/389,837    -   Ser. No. 16/985,093    -   Ser. No. 16/583,967    -   62/828,133    -   Ser. No. 16/542,577    -   62/820,190    -   Ser. No. 16/523,501    -   Ser. No. 15/411,424

BACKGROUND OF THE INVENTION Field of the Art

The disclosure relates to the field of mobile device technology,specifically to the field of integrating cloud-based callback managementsystems with onboard software on a mobile device.

Discussion of the State of the Art

Typically, if a client calls a business, voice prompt menu choicesenable the calling client to identify an issue for which the clientrequires service and the client is then queued for a service agentcapable of handling the identified issue. As such, it is expected thatclients who identify the purpose of their call as a “billing issue” willbe queued for, and connected to, a service representative with theability to handle billing issues. Similarly, it is expected that clientswho identify the purpose of their call as a “customer service issue”will be queued for, and connected to, a service representative with theability to handle customer service issues.

There are problems with existing communications systems, such as contactcenters, including the following two problems. First, the voice promptmenus that are used to channel callers to the queue for the appropriategroup of service agents are frustrating to clients. It takes significanttime to navigate the layered menus of voice prompts.

Second, waiting on-hold while a connection, be it a phone call, webchat, video conference, or other interaction type, is maintained inqueue for connection to a service agent is also frustrating to clients.

In an effort to reduce customer exacerbation caused by having tomaintain a connection while on-hold in queue, secondary queue systemshave been developed. A typical secondary queue system obtains atelephone number at which the calling client can be reached when aservice representative is available (i.e., a call back number). Theclient disconnects, and then, at the proper time, a call back systemestablishes a connection to the client utilizing the call back numberand couples the client to an available representative without waitingon-hold in queue. One exemplary system is disclosed in U.S. Pat. No.6,563,921 to Williams et al. which is commonly assigned with the presentapplication.

While such a system may make the experience of waiting for a connectionto a service representative less frustrating, it does not address theinconvenience of having to navigate a slow and complicated voice promptmenu to enter the queue.

Additionally, as the gig economy compounds and people tend to get busierin the professional workplace, a solution is needed on their personaldevice to help prioritize returning communications with personalcontacts, business relations, or solicitors.

What is needed is a system and various methods for providing integrationof a cloud callback platform with mobile device software so thatcallback functionality and prioritization becomes a transparent andconsistent feature across interaction modes through the mobile deviceecosystem.

SUMMARY OF THE INVENTION

Accordingly, the inventor has conceived and reduced to practice, asystem and methods for mobile device active callback prioritization,utilizing an enhanced callback prioritization engine operating on auser's mobile device for integration through the operating system andsoftware applications operating on the device, wherein the enhancedcallback prioritization engine receives intercepted data or voicemessages sent to the mobile device, retrieves and aggregates datarelated to the assigned messages, inputs the assigned data message andaggregate data into one or more machine learning algorithms wherein thealgorithms may analyze the input data, the results of the analysis maybe used to compute a priority score for the assigned data message, andgenerates a callback list from the computed prioritization score. Thepriority score is in part based on 3^(rd) party application data relatedto the data or voice messages providing context to the machine learningalgorithms.

According to a first preferred embodiment, a mobile device with activecallback prioritization is disclosed, comprising: a processor, a memory,and a plurality of programming instructions stored in the memory andoperable on the processor; a callback integration engine comprising asubset of the plurality of programming instructions that, when operatingon the processor, cause the processor to: receive a data or voicemessage, the data or voice message comprising at least onecharacteristic; produce a callback object in memory comprisinginformation associated with the data or voice message received; and sendthe callback object to an enhanced callback prioritization engine; andthe enhanced callback prioritization engine comprising a subset of theplurality of programming instructions that, when operating on theprocessor, cause the processor to: receive the callback object from thecallback integration engine; retrieve and aggregate application datarelated to the data or voice message; use the callback object and theaggregated application data as inputs into one or more machine learningalgorithms, wherein the algorithms analyze the callback object'sinformation and the aggregated application data to determine the contextand urgency associated with the data or voice message; for each callbackobject, compute a priority score based at least upon the results of theanalysis; and use the computed priority score, the callback object data,and the data or voice message to generate a callback list.

According to a second preferred embodiment, a method for active callbackprioritization is disclosed, comprising the steps of: receiving a dataor voice message, the data or voice message comprising at least onecharacteristic; producing a callback object in memory comprisinginformation associated with the data or voice message received;retrieving and aggregating application data related to the data or voicemessage; using the callback object and the aggregated application dataas inputs into one or more machine learning algorithms, wherein thealgorithms analyze the callback object's information and the aggregatedapplication data to determine the context and urgency associated withthe data or voice message; for each callback object, computing apriority score based at least upon the results of the analysis; andusing the computed priority score, the callback object data, and thedata or voice message to generate a callback list.

According to various aspects: wherein the application data is retrievedusing application programming interfaces; wherein the callback listcomprises a smart reply message; wherein a user confirmed callback listis generated and executes the confirmed callback items on the list;wherein the application data comprises data from communication, socialmedia, financial, gaming, and productivity applications; and wherein themachine learning algorithms comprise natural language processing.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several aspects and, together withthe description, serve to explain the principles of the inventionaccording to the aspects. It will be appreciated by one skilled in theart that the particular arrangements illustrated in the drawings aremerely exemplary, and are not to be considered as limiting of the scopeof the invention or the claims herein in any way.

FIG. 1 is a block diagram illustrating an exemplary system architecturefor a mobile device connected to a cloud callback platform via a publicswitch telephone network and the Internet, according to an embodiment.

FIG. 2 is a block diagram illustrating an exemplary mobile device with acallback integration engine and callback prioritization engine operatingat the operating system level, according to an embodiment.

FIG. 3 is a block diagram illustrating an exemplary mobile device with acallback integration engine and callback prioritization engine operatingat the device firmware level, according to an embodiment.

FIG. 4 is a block diagram illustrating an exemplary system architecturefor a callback prioritization engine, according to an embodiment.

FIG. 5 is a user interface diagram illustrating an exemplary callbacklist displayed on a mobile device with integrated callback features,according to an embodiment.

FIG. 6 is a user interface diagram illustrating an exemplary incomingcall on a mobile device with integrated callback features, according toan embodiment.

FIG. 7 is a user interface diagram illustrating an exemplary emailapplication operating on a mobile device with integrated callbackfeatures, according to an embodiment.

FIG. 8 is a method diagram illustrating an exemplary incoming call flow,according to an embodiment.

FIG. 9 is a method diagram illustrating an exemplary callback workflowonce a user selects a callback for an incoming call, according to anembodiment.

FIG. 10 is a method diagram illustrating an exemplary callbackprioritization workflow once a data or voice message is received,according to an embodiment.

FIG. 11 is a block diagram illustrating an exemplary system architecturefor an exemplary mobile device embodiment with an enhanced callbackprioritization engine operating at the operating system level andconnected to a cloud callback platform, according to an embodiment.

FIG. 12 is a block diagram illustrating an exemplary system architecturefor an enhanced callback prioritization engine, according to anembodiment.

FIG. 13 is a method diagram illustrating several app integrations forenhancing callback prioritization.

FIG. 14 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 15 is a block diagram illustrating an exemplary logicalarchitecture for a client device.

FIG. 16 is a block diagram showing an exemplary architecturalarrangement of clients, servers, and external services.

FIG. 17 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a system andmethods for mobile device active callback prioritization, utilizing anenhanced callback prioritization engine operating on a user's mobiledevice for integration through the operating system and softwareapplications operating on the device, wherein the enhanced callbackprioritization engine receives intercepted data or voice messages sentto the mobile device, retrieves and aggregates data related to theassigned messages, inputs the assigned data message and aggregate datainto one or more machine learning algorithms wherein the algorithms mayanalyze the input data, the results of the analysis may be used tocompute a priority score for the assigned data message, and generates acallback list from the computed prioritization score. The priority scoreis in part based on 3^(rd) party application data related to the data orvoice messages providing context to the machine learning algorithms.

One or more different aspects may be described in the presentapplication. Further, for one or more of the aspects described herein,numerous alternative arrangements may be described; it should beappreciated that these are presented for illustrative purposes only andare not limiting of the aspects contained herein or the claims presentedherein in any way. One or more of the arrangements may be widelyapplicable to numerous aspects, as may be readily apparent from thedisclosure. In general, arrangements are described in sufficient detailto enable those skilled in the art to practice one or more of theaspects, and it should be appreciated that other arrangements may beutilized and that structural, logical, software, electrical and otherchanges may be made without departing from the scope of the particularaspects. Particular features of one or more of the aspects describedherein may be described with reference to one or more particular aspectsor figures that form a part of the present disclosure, and in which areshown, by way of illustration, specific arrangements of one or more ofthe aspects. It should be appreciated, however, that such features arenot limited to usage in the one or more particular aspects or figureswith reference to which they are described. The present disclosure isneither a literal description of all arrangements of one or more of theaspects nor a listing of features of one or more of the aspects thatmust be present in all arrangements.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an aspect with several components in communication witheach other does not imply that all such components are required. To thecontrary, a variety of optional components may be described toillustrate a wide variety of possible aspects and in order to more fullyillustrate one or more aspects. Similarly, although process steps,method steps, algorithms or the like may be described in a sequentialorder, such processes, methods and algorithms may generally beconfigured to work in alternate orders, unless specifically stated tothe contrary. In other words, any sequence or order of steps that may bedescribed in this patent application does not, in and of itself,indicate a requirement that the steps be performed in that order. Thesteps of described processes may be performed in any order practical.Further, some steps may be performed simultaneously despite beingdescribed or implied as occurring non-simultaneously (e.g., because onestep is described after the other step). Moreover, the illustration of aprocess by its depiction in a drawing does not imply that theillustrated process is exclusive of other variations and modificationsthereto, does not imply that the illustrated process or any of its stepsare necessary to one or more of the aspects, and does not imply that theillustrated process is preferred. Also, steps are generally describedonce per aspect, but this does not mean they must occur once, or thatthey may only occur once each time a process, method, or algorithm iscarried out or executed. Some steps may be omitted in some aspects orsome occurrences, or some steps may be executed more than once in agiven aspect or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other aspects need notinclude the device itself.

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular aspects may include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of various aspects in which, for example,functions may be executed out of order from that shown or discussed,including substantially concurrently or in reverse order, depending onthe functionality involved, as would be understood by those havingordinary skill in the art.

Definitions

“Callback” as used herein refers to an instance of an individual beingcontacted after their initial contact was unsuccessful. For instance, ifa first user calls a second user on a telephone, but the second userdoes not receive their call for one of numerous reasons includingturning off their phone or simply not picking up, the second user maythen place a callback to the first user once they realize they missedtheir call. This callback concept applies equally to many forms ofinteraction that need not be restricted to telephone calls, for exampleincluding (but not limited to) voice calls over a telephone line, videocalls over a network connection, or live text-based chat such as webchat or short message service (SMS) texting. While a callback (andvarious associated components, methods, and operations taught herein)may also be used with an email communication despite the inherentlyasynchronous nature of email (participants may read and reply to emailsat any time, and need not be interacting at the same time or while otherparticipants are online or available), the preferred usage as taughtherein refers to synchronous communication (that is, communication whereparticipants are interacting at the same time, as with a phone call orchat conversation).

“Callback object” as used herein means a data object representingcallback data, such as the identities and call information for a firstand second user, the parameters for a callback including what time itshall be performed, and any other relevant data for a callback to becompleted based on the data held by the callback object.

Conceptual Architecture

FIG. 1 is a block diagram illustrating an exemplary system architecture100 for a mobile device 101 connected to a cloud callback platform 110via a public switch telephone network 103 and the Internet 102,according to an embodiment. According to the embodiment, callback cloud110 may receive requests via a plurality of communications networks suchas a public switched telephone network (PSTN) 103 or the Internet 102.These requests may comprise a variety of communication and interactiontypes, for example including (but not limited to) voice calls over atelephone line, video calls over a network connection, or livetext-based chat such as web chat or short message service (SMS) textingvia PSTN 103. Such communications networks may be connected to aplurality of mobile devices 101 such as a user's smartphone or similarmobile device, according to the particular architecture of communicationnetwork involved. Mobile devices 101 may be connected to respectivecommunications networks via a variety of means, which may includetelephone dialers, VOIP telecommunications services, web browserapplications, SMS text messaging services, or other telephony or datacommunications services. It will be appreciated by one having ordinaryskill in the art that such means of communication are exemplary, andmany alternative means are possible and becoming possible in the art,any of which may be utilized as an element of system 100 according tothe invention.

When needed (for example, when a user manually requests a callback orwhen configured settings determine that a callback is needed), a user'smobile device 101 creates a session with a cloud callback platform 110with a profile manager 111, resulting in a callback being required.Profile manager 111 receives initial requests to connect to callbackcloud 110, and forwards relevant user profile information to a callbackmanager 113, which may further request environmental context data froman environment analyzer 112. Environmental context data may include (forexample, and not limited to) recorded information about when a user maybe suspected to be driving or commuting from work (as may be determinedfrom device information, such as whether a “do not disturb whiledriving” feature has been enabled, or if the mobile device 101 iscurrently connected to a car audio system), or if the user may be busyfor another reason, such as if they are running or working out (as maybe determined from device sensor data), for example, and may be parsedfrom online profiles or online textual data, using an environmentanalyzer 112.

A callback manager 113 centrally manages all callback data, creating acallback object which may be used to manage the data for a particularcallback, and communicates with an interaction manager 114 which handlesrequests to make calls and bridge calls, which go out to a media server115 which actually makes the calls as requested. In this way, the mediaserver 115 may be altered in the manner in which it makes and bridgescalls when directed, but the callback manager 113 does not need toadjust itself, due to going through an intermediary component, theinteraction manager 114, as an interface between the two. A media server115, when directed, may place calls and send messages, emails, orconnect voice over IP (“VoIP”) calls and video calls, to users over aPSTN 103 or the Internet 102. Callback manager 113 may work with auser's profile as managed by a profile manager 111, with environmentalcontext from an environment analyzer 112 as well as (if provided) EWTinformation for any callback recipients (for example, contact centeragents with the appropriate skills to address the callback requestor'sneeds, or online tech support agents to respond to chat requests), todetermine an appropriate callback time for the two users (a callbackrequestor and a callback recipient), interfacing with an interactionmanager 114 to physically place and bridge the calls with a media server115.

The cloud callback platform 110 may further include an applicationprogramming interface (API) manager 116 and a callback prioritizationengine 117, according to an embodiment. The API manager 116 may be usedto expose or connect with software related to a callback requester and acallback recipient. For example, API manager 116 may be used to connectwith email, messaging, telephone, or other voice communicationapplications present on a mobile device in order to retrieve text-baseddata related to emails, text messages such as SMS and from othermessaging applications, or voice data such as voicemails. Voice datasuch as, for example, a voice mail may be transcribed into a text-baseddata message by the cloud callback platform 110. The retrievedtext-based data messages may be sent to a prioritization engine 117which may identify one or more characteristics of the text-based datamessages to route the data messages to an appropriate lobby. Theprioritization engine 117 may use the environment analyzer to parse oranalyze the retrieved text-based data messages. Analysis of thetext-based data messages may be used to determine or infer a variety ofmetrics pertaining to a callback requester and at least one callbackrecipient, such as (but not limited to) relationships, intent, subjectmatter, and sentiment. Based on the analysis of the text-based datamessages, the callback prioritization engine 117 may produce priorityscore for each text-based data message which can be used to orderlobbies (callback groups) in order of priority.

FIG. 2 is a block diagram illustrating an exemplary mobile device 101with a callback integration engine 201 and callback prioritizationengine 202 operating at the operating system level, according to anembodiment. According to the embodiment, a mobile device 101 comprisesseveral hardware and software components operating at various levels toprovide various functions. At the most basic hardware level 210, thephysical hardware of the mobile device 101 may comprise a processor andmemory 211 that provide standard computing device functionality asdescribed in detail below, with reference to FIGS. 14-17 (groupedtogether here for the sake of clarity, it should be appreciated thatprocessor/memory may or may not be part of the same hardware component,such as a system-on-a-chip), a baseband processor 212 that managesradio-based communication functions such as cellular or Wi-Ficonnectivity, as well as any of a number of hardware sensors 213 such as(for example, including but not limited to) an accelerometer to detector measure device movement, gyroscope for detecting or measuring deviceorientation, barometer for measuring ambient environment conditions, orglobal positioning system (GPS) receiver for geolocating the device.

An operating system (OS) 220 comprises the main software operating onmobile device 101 and providing various software-based functions such assoftware applications and communications. Various software applications221 operating within (for example in an application layer not shown herebut as is commonly used in various computing devices according to thestandard OSI process model of computing systems) on the mobile device101 may expose and provide access to, or interaction with, varioushardware and sensor features such as to enable a user to view orcalibrate sensor readings.

According to the embodiment, a callback integration engine 201 maycomprise local (that is, operating on the mobile device) functionalitysimilar to a cloud-based callback manager 113 described above, and mayoperate as an application or feature at the operating system level 220,running at a similar privilege level and having similar access tohardware and software functions as other applications 221 operating onmobile device 101. This enables easy installation or removal of callbackintegration engine 201, as it may be readily distributed via similarmethods as any other software application (for example, via anapplication store or similar download portal). However, this mayrestrict the capabilities of callback integration engine 201, as it willhave only indirect access to hardware features (as it will only be ableto access whatever features are exposed by the operating system) and inmany cases may be “sandboxed”, and prevented from directly accessingother software or information on the mobile device 201 (for example, ina sandboxed software environment such as APPLE IOS™ or similar).

According to the embodiment, a callback prioritization engine 202 maycomprise local (that is, operating on the mobile device) functionalitysimilar to a cloud-based prioritization engine 117 described above, andmay operate as an application or feature at the operating system level220, running at a similar privilege level and having similar access tohardware and software functions as other applications 221 operating onmobile device 101. The callback prioritization engine 202 may beincluded within the same application as the callback integration engine201 or it may be packaged as it own application with built infunctionality to communicate with and share data with a separatecallback integration engine 201. This enables easy installation orremoval of callback prioritization engine 202, as it may be readilydistributed via similar methods as any other software application (forexample, via an application store or similar download portal). However,this may restrict the capabilities of callback prioritization engine202, as it will have only indirect access to hardware features (as itwill only be able to access whatever features are exposed by theoperating system) and in many cases may be “sandboxed”, and preventedfrom directly accessing other software or information on the mobiledevice 201 (for example, in a sandboxed software environment such asAPPLE IOS™ or similar).

Callback prioritization engine 202 may receive from the callbackintegration engine 201 a callback object associated with intercepteddata or voice messages. Callback prioritization engine 202 may intercept(e.g., receive, hook, access, or otherwise obtain) voice messages (i.e.,voicemail) and text-based data messages such as (but not limited to)short message service (SMS), emails, and group chats sent to the mobiledevice 101 and route them to an appropriate lobby (e.g., group) basedupon at least one characteristic of the message. In an embodiment, themobile device may transcribe voice messages into text-based data messagefor further processing. Each text-based data message and voice message(or its transcription) may have one or more characteristics, such asindications of source, destination, data message subject matter, enduser identifiers or account identifiers. Once a data message has beenassigned to a lobby other data associated with the data message may beaggregated and linked to the callback object. For example, a first emailsent to a mobile device 101 may be intercepted and assigned to a lobby,and then the callback prioritization engine 202 can identify andretrieve other emails, text messages, or group chats associated with thefirst email in order to provide more context for the first emailmessage. In this example, other communications exchanged between thefirst email recipient and the first email sender (such as an emailthread, text messages, or voicemails) may be aggregated and added to thecallback object associated with the first email. The aggregated data maythen be fed into one or more machine learning algorithms that analyzethe aggregated data and can generate for each data message in the lobbya priority score. The priority score may be used to create a hierarchyor priority list of callback objects such that each callback may bescheduled and executed in order of highest priority to lowest priority.

FIG. 3 is a block diagram illustrating an exemplary mobile device 101with a callback integration engine 201 and callback prioritizationengine 202 operating at the device firmware level, according to anembodiment. According to the embodiment, a firmware level 320encompasses low-level program code that operates “below” the operatingsystem 220, operating directly on hardware components of mobile 101, andcomprises such features as a baseband controller 321 that controlsfeatures of baseband processor 212 and the operation of which is fullytransparent to the user (that is, a user of mobile device 101 does notsee or interact with firmware, and many users may not even be aware ofits existence or capabilities).

According to the embodiment, a callback integration engine 201 and acallback prioritization engine 202 may operate as a firmware componentthat directly interfaces with hardware features of mobile device 101,enabling direct control as needed while exposing desired functionalityto the operating system 220 in a transparent manner (that is, theoperating system may only know that a feature is available, and may notbe able to determine that the feature is provided by the callbackintegration engine 201 or callback prioritization engine 202 ratherthan, for example, the baseband controller 321). This enables moredirect control over device functions, such as call and data messagerouting and hardware information such as sensor readings (as may be usedin callback workflow processing), and allows the callback integrationengine 201 to intercept incoming calls and radio information (e.g., datamessages) before it reaches the operating system, so data may bemanipulated and additional features may be integrated prior topresentation to the OS. This also prevents accidental removal ofcallback integration engine 201 and callback prioritization engine 202,as a user may have no control over device firmware and anythingoperating at or below the OS level 220 would inherently haveinsufficient access to modify firmware. This further enables callbackintegration engine 201 and callback prioritization engine 202 to accesshardware features that may be restricted or completely prevented whenoperating within the OS layer 220, for example hardware sensor orgeolocation information that may be incorporated into a callbackworkflow.

FIG. 4 is a block diagram illustrating an exemplary system architecture400 for a callback prioritization engine 202, according to anembodiment. According to the embodiment, a callback prioritizationengine 202 may receive a data message with a characteristic. The datamessage can part of sessions that include asynchronous text, voicemessage, or data message (e.g., not a live voice conversation such as atelephone call) communications between and end user and a computingdevice (e.g., mobile device, contact center agent [e.g., a person, bot,or combination thereof], etc.). In some examples, callbackprioritization engine 202 can receive voice messages that have been sendas audio messages (voicemail) or converted into text messages. Thesessions can occur continuously in a conversational manner, or can beintermittent, e.g., lasting over hours, days or weeks with dormantperiods between communications. The callback prioritization engine 202may receive data messages and can identify characteristics of those datamessages.

Based upon the characteristics of the data messages, the lobbyassignment mechanism 401 can assign the data message to a default lobbyconstruct (group) or can bypass the default lobby construct and insteadassign the data message to the destination lobby construct. For example.The lobby assignment mechanism 401 can determine that thecharacteristics of a data message do not match or correlate tocharacteristics of other data messages that have been assigned to one ormore destination constructs. When the lobby assignment mechanism 401does identify a match or correlation between a (new incoming) datamessage and a data message assigned to a particular destination lobbythe lobby assignment mechanism 401 can assign the new incoming datamessage the same destination lobby. The lobby constructs holdcollections of data messages corresponding to communications or sessionsbetween two or more computing devices (e.g., an end user mobile deviceand a contact center agent computing device or coworker computingdevice). The plurality of lobby constructs can be identical instances ortypes of structures, differing only in the collections of data messagesthey hold and the teams to which they are assigned. For example, therecan be exactly one lobby construct assigned per defined team of contactcenter agent computing devices to attend to defined issues, such ascustomer complaints or service issues. Similarly, there can be exactlyone lobby construct assigned per defined teams (e.g., marketing, sales,customer retention, managerial staff, etc.) within a business structureor a lobby construct may be assigned to a single person or entity. Thecollections of data messages for this team may be held by a single lobbyconstruct.

Once a data message has been assigned to a lobby construct the callbackprioritization engine 202 may identify and aggregate other data relatedto the data message. For example, if an assigned data message is part ofan ongoing session (such as an email thread, text or group messagingconversation) then a data aggregator 402 may retrieve the other messagescontained within the session and append or link the other messages tothe assigned data message. Additionally, if the assigned data message isa stand alone message (i.e., sent with no connection to a prior session)then the callback prioritization engine 202 may scan for and identifyother communications that may be related to the assigned data message.Other communications may be identified as related based upon one or morecharacteristics of the assigned data message. For example, an end usermay receive an email asking for a response of some sort from a coworkerwhich may be related to previous communications exchanged between theend user and the coworker, but not contained within the same thread asthe previous communications. In this example, the callbackprioritization engine 202 may be able to identify the previouscommunications as related to the original email asking for a responseand append or link them to the assigned data messages generated from theresponse email. As another example, an end user mobile device 101 mayreceive a voicemail from an individual or entity (business) asking theend user to callback when available. The voicemail may be transcribed toa text data message and have one or more characteristics, such as thephone number or other identifier associated with the voicemail, and thedata aggregator 402 can use the voicemail characteristics to identify anemail (or other data message) that may be associated with the voicemail.For example, an employee of a company may send to a second employee avoicemail requesting a callback and further, the two employees may havebeen emailing or text messaging each other earlier in the day or week.These emails or text messages may be linked to the transcribed voicemailby the data aggregator 402. In this way the data aggregator 402 mayidentify and aggregate data correlating with a data message assigned toa lobby construct.

The assigned data message and any linked and aggregated data may then besent as input to a machine learning (ML) engine 403 for processing andanalysis. The ML engine 403 may comprise one or more machine learningalgorithms such as, for example, a natural language processing (NLP)neural network. The NLP may be used for text classification functionsincluding, but not limited to, sentiment analysis, topic categorization,language recognition, and intent analysis. The NLP may be used toperform sentiment analysis on the assigned data message and anyavailable aggregated data linked to the data message. Sentiment analysismay be used to determine if the content of a data message (and itsaggregate data) is positive, negative, or neutral. This could be used toclassify data messages to better respond to inquiries or requests. TheNLP may be used for topic categorization to determine which category (orlobby construct) a data message belongs to, essentially this constitutesa robust, automatic method of tagging and organizing the content of adata message or its aggregate data. This may be used to automaticallyseparate data messages within a lobby construct by topic. The NLP may beused to understand the intent of a data message or its aggregate data bydetermining the underlying intention of a given data message.Additionally, the NLP may be used to find and identify relationshipsbetween a caller and callee, or a message recipient and a messagesender, or between and among a group of people (i.e., multiple people inan email thread or group text message). Using the identifiedrelationships, callback prioritization engine 202 or callbackintegration engine 201 may generate personalized smart replies based onthe identified relationships and other information (intent, sentiment,topic) derived from the NLP. The relationships may be used to determinethe priority of an assigned data message within a lobby construct.

In an embodiment, the ML engine 403 may also comprise one or more audiomachine learning algorithms which may be used for audio classificationand speech recognition functions. Audio classification ML algorithms maybe used when a voice messages (voicemail) is received by mobile device101 or by callback prioritization engine 202. The audio algorithm may beused to identify a speaker by taking a snippet of the voice message tocreate a unique identifier, like a voice fingerprint. Then thefingerprint may be input into a neural network that can match the uniqueidentifier to a known or previously heard voice. This may allow callbackprioritization engine 202 to identify callers who may be calling from anew device or using a new identifier that may not be known by thecallback integration engine 201. Furthermore, an audio ML algorithm maybe utilized for speech recognition functions such as voice to texttechnology (i.e., transcribing a voicemail into a text data message) orto provide voice assistant interaction (e.g., in a contact centerenvironment whereby a customer may communicate with a bot or scriptvoice assistant).

There are various benefits of having a ML engine 403 operating locallyon a mobile device compared to operating in a cloud environment whichthe mobile device may connect to. Operating the ML engine 403 on amobile device 101 protects against low latency. As most ML algorithmsmay be operating in real-time, network speed may be too slow to transmita live data feed from the device, to the cloud callback platform 110,and then back to the device. Furthermore, running the ML engine 403locally on a mobile device 101 removes the risk of low or no networkconnections halting the ML processes. In a mobile device 101 the MLengine 403 may still work when run on device, even if low or noconnectivity. Operating the ML engine 403 locally on a mobile device 101facilitates privacy over the mobile device user's data. ML needs data tobuild a useful model and to make useful inferences derived from themodel, however, some data may be sensitive (e.g., personal identifyinginformation, regulated data). The callback prioritization engine 202 canuse and secure data on the device, without having to transmit anysensitive data.

The ML engine 403 may forward the results of the analyses (e.g.,inferences, intent, sentiment, topic, etc.) of the various ML algorithmsto a prioritization module 404. The prioritization module 404 may alsoreceive, from the lobby assignment mechanism 401, the assigned lobbyconstruct(s) comprising at least one or more assigned data messages. Theprioritization module 404 may then use the results of the analyses(e.g., inferences, intent, sentiment, topic, etc.) to generate apriority score for each data message assigned to a lobby construct. Forexample, if multiple assigned messages within a construct have beenidentified to belonging to the same topic, then the most recent datamessage pertaining to the shared topic may be given a higher priorityscore and thus the callback associated with that data message would beexecuted before the previously received data messages. For example, acable company service agent may be assigned to a lobby construct whichhas been assigned twenty five data messages, and of those data messagestwenty are about a loss of cable service. In this example, the ML engine403 may be able to identify that each of those twenty data messagesoriginated from the same area code and therefore there must be someservice disruption affecting that area code. The prioritization module404 can use this information to move the first of the twenty people upin priority so that the problem with their cable service may beaddressed. If the cable service agent is able to resolve the issue forthe first of twenty data messages relating to cable service loss, thenthe callback prioritization engine 202 may be able to send an automatedmessage to the other of the customers associated with the remainingnineteen data messages providing information about the loss of cableservice. For example, an automated message may be sent which says, “Isyour call about loss of cable service? If so, we are aware and yourservice should be restored soon. Would you like to request a callback?”.If the receiver of the automated message replies with a callbackrequest, then the data message may remain in the lobby construct,otherwise the data message will be removed from the lobby construct.

Intent and sentiment analysis results may be used to determine priorityas well. For example, the sentiment of one data message in a lobbyconstruct may be determined to be positive, whereas the sentiment ofanother data message in same lobby construct may be urgent, or negative.A data message with an urgent or negative sentiment may be given ahigher priority score than a data message with a positive sentiment inorder to account for the urgency of the data message, or to quicklyrespond to the negative sentiment data message in order to changesentiment of the data message sender to neutral or positive.Additionally, the intent of a data message may be used to determine thepriority of response. For example, a data message in a lobby constructmay indicate that there was no intent for a callback or response,whereas another data message in the lobby construct may have an inferredintent for a callback. In this case, the data message with the inferredintent for a callback will be given a higher priority score than thedata message with little to no inferred callback intent.

Based upon the priority score of each data message in a lobby, theprioritization module 404 may reorder the lobby in a hierarchal mannerwhere the highest priority data message is at the top of the lobby andthe other data messages are sorted in descending order of priority. Theprioritization module 404 may also produce a callback list based uponthe sorted lobby. The callback list may include the original datamessage, any aggregate data, and data contained within the callbackobject associated with each data message. For example, the callback listmay display the original data message, a scheduled callback time, andthe name of the callback recipient. The callback list may be sent to themobile device 101 (or any computing device) for display and interactionby the mobile device user.

FIG. 5 is a user interface diagram illustrating an exemplary callbacklist 500 displayed on a mobile device 101 with integrated callback andprioritization features, according to an embodiment. According to theembodiment, a mobile device with integrated callback and prioritizationfeatures provided by a callback prioritization engine 202 which maygenerate a callback list from received data messages or voice messages.For example, a mobile device 101 user may be unable to respond to aphone (or voice over internet protocol, “VoIP”) call, email, text, orother functionality. A user may be unable to respond for a plurality ofreason such as, for example, the user may already be in a call, the useris in a meeting, the user is driving, or the user is otherwise disposed(e.g., working out, doing yard work, etc.). In these instances, thecallback integration engine 201 may create a callback object for eachreceived data message or phone call. The callback objects may be sent toa callback prioritization engine 202 where the callback objects may beappended with additional data relating to the received data message orphone call. The data messages or associated callback objects may beassigned to a lobby construct (i.e., callback group). The data messagesmay then be analyzed via one or more ML algorithms to further provide,determine, generate, or infer context information that may be used tocompute priority scores for each data message or callback object in alobby construct. A callback list 500 may be generated from the lobbyconstructs, the data messages or callback objects contained with a lobbyconstruct, and the computed priority scores to produce a hierarchalcallback list based on priority. In this way, when a mobile device 101user, who was otherwise unable to respond to the data messages and phonecalls, is able to interact with the mobile device 101 (s)he may be ableto view an automatically generated callback list with tentative callbacktimes scheduled.

In an embodiment, the callback list may be displayed on a mobile device101 allowing a mobile device user to view and interact with the callbacklist 500. The callback list 500 may display one or more data messages501. For example, the message(s) displayed may be the data message thatwas received while the user was unavailable and any other aggregate datathat may be useful for the device user. In some embodiments, the mobiledevice 101 may be able to automatically generate smart reply messagesthat can be sent as a reply to the received data messages or voicemessages (voicemails, missed calls) and displayed in the message(s) 501display. In addition to the displaying the assigned and sorted message501 the callback list 500 may also display a tentative, scheduledcallback time 502 determined by the callback integration engine 201.Also present in this embodiment, are options to delete 504 and move 503the scheduled callback from the callback list. For example, a mobiledevice 101 user may decide to send an email in response to the seconditem on the callback list 500 instead of proceeding with a scheduledcallback, in this instance the user may choose to delete 504 thecallback from the callback list. Another option the mobile device 101user may choose is to move 503 the listed callback items. This may bedone by selecting a move 503 button on one callback list item and thenselecting another move 503 button on a different callback list itemfacilitating a swap of callback list items in the hierarchy. Another wayto move 503 callback list items may be by drag-and-dropping callbackitems. In this way, one callback item may be moved without having toswap places with another callback item. If an automated smart replymessage has be generated and displayed in the callback list, the mobiledevice 101 user may review and amend the smart reply message within thecallback list and then select an option to send the smart reply message.After a smart reply message has been sent, the user may choose to deleteor maintain the callback item associated with the smart reply. A mobiledevice 101 user may choose to confirm 505 the callback list 500 withoutmaking any changes, or the user can make alterations (move, delete,swap, etc.) to the callback list 500 and then confirm 505 it. If aconfirmation is made without any alterations, the callback integrationengine 201 may execute the callback list 500 at the displayed scheduledtimes 502. If alterations are made to the callback list 500, then thecallback integration engine 201 may need to reschedule callback itemsbased upon the alterations implemented by the mobile device 101 user andthen execute the scheduled callbacks.

FIG. 6 is a user interface diagram illustrating an exemplary incomingcall 600 on a mobile device 101 with integrated callback features,according to an embodiment. According to the embodiment, a mobile device101 with integrated callback features provided by a callback integrationengine 201 may natively incorporate callback functionality into standardfunctions such as receiving a phone (or voice over Internet protocol,“VoIP”) call, email (as described below with reference to FIG. 7 ), orother functionality. When a call 600 is received, the native answerprompt may present the user with the usual options to answer 601 ordecline 603 the call, with normal functionality (generally, eitheranswering the call and starting an interaction, or declining the calland sending it to voicemail or an automated prompt).

Callback integration engine 201 may present an additional prompt tosetup a callback 602, either through OS-based software integration withthe callback integration engine operating at the OS layer 220, or as abase-level firmware feature that is natively recognized and exposed bythe OS while the callback integration engine 201 operates at thefirmware level 320. This added option 602 may be used to automaticallyrequest or schedule a callback, for example by providing a message tothe caller requesting they call back at a predetermined time (forexample, based on known availability from a user's on-device calendar),or by engaging with a cloud callback platform 110 to automaticallyarrange a callback that connects both participants. Whether or not thecallback prompt 602 is presented may be configurable, such as byincorporating trust lists or zones that determine what callers may beeligible for a callback (similar to a “favorite contacts” list that maybe able to call the user even when a do-not-disturb feature is enabled),or context-based configuration such as to provide a callback prompt whenthe user is in a meeting or otherwise scheduled as “busy” in theircalendar, or when the user is driving.

FIG. 7 is a user interface diagram illustrating an exemplary emailapplication 700 operating on a mobile device 101 with integratedcallback features, according to an embodiment. According to theembodiment, an email message or contact may be augmented with callbackfunctionality provided by callback integration engine 201, presented asan option 701 to setup an automated callback from within an emailapplication 700. This feature may incorporate information from thecurrent email such as a topic 703, timing 704, or contact information702, as well as any additional device, context, user, or otherinformation that may be relevant (for example, calendar information orother emails outside the current message or thread), and cause acallback object to be generated. This callback object may then be usedto automatically schedule and execute a callback by contacting theparticipants 702 of the email (and it should be appreciated that thisneed not be limited to two participants, and may be used to createautomated conference calls), and then bridge the individual calls toeach participant to complete the interaction by connecting theparticipants together into a single call.

Callbacks may be scheduled according to a variety of criteria, including(but not limited to) user availability as determined from preconfiguredsettings or known context (for example, calendar or email informationsuch as invitation responses or verbal commitments in messages that maynot have been separately entered into a calendar), user activity basedon device information such as network or sensor data (such as if thedevice is paired to a car audio system, indicating the user is driving,or if there is significant accelerometer data that might indicate theuser is in the middle of an exercise activity). Callbacks may then bescheduled to occur when the user is available or no longer indisposed,and may also incorporate availability on the part of the caller by (forexample) providing them with a selection of callback options to fromwhich to select a specific callback time. In addition to providing acallback selection on an incoming call prompt, the callback function maybe exposed in other areas throughout the device's OS and applications,such as from within voicemail messages (to setup an automated callbackwith the caller that left the voicemail), social media apps (to setupautomated callbacks with other users), or potentially any applicationoperating on the device 101 (such as to setup an automated callback fortechnical support). This integrated callback operation may be consistentthroughout the device's software, providing a native user experiencethat blends seamlessly with the other features and elements of thedevice's operating system and applications.

To generate, schedule, and execute callbacks, a callback object iscreated on the user's device 101 to represent the callback informationsuch as scheduling, context information, user and caller information,and any additional data pertinent to the callback (for example, relatedinteractions such as previous calls or emails exchanged with the caller,or a known call intent based on available information from the userand/or caller, such as email transcripts or voicemail messages). Thiscallback object may be created and maintained on-device, operatinglocally within the callback integration engine 201 or within anapplication or feature of the operating system 220 of the user's device101, enabling full callback functionality regardless of any connectionto, or availability of, a cloud callback platform 110. In otherarrangements, callback objects may be cloud-based to provide acentralized or software-as-a-service (SaaS) operation mode, for exampleto provide tiered or subscription-based callback functionalities offeredby a callback cloud through handling of callback objects on behalf ofusers.

FIG. 8 is a method diagram illustrating an exemplary incoming call flow,according to an embodiment. According to the embodiment, a call isreceived 801 at a mobile device 101. A callback integration engine 201then analyzes the information available for the call 802, such as (forexample, including but not limited to) caller ID, caller and recipienttime zones, or whether the caller is a member of a trust zone in theuser's settings or contact information (or a trust zone not configuredby the user, such as a corporate trust zone for coworkers andcolleagues). If the caller is eligible (that is, if they have sufficienttrust or if their call is determined to be valuable to the user, such asa call from a technical support number for a company the user recentlycontacted), a callback token is associated with the call data 803. Ifthe user is untrusted, such as a blocked user or a suspected “spam”number, the call may be passed to the OS layer unmodified 804. Whenreceived by the OS layer, the call is then displayed as an incomingnotification as usual 805, with a callback token (if available) used toplace an additional callback button within the interface using thenative OS and call notification user experience (UX) design. Thisprovides an integrated automated callback functionality that istransparently incorporated into device features in a manner consistentwith the operating system's UX and familiar to the user, blending thenew functionality with the rest of the device features. It should befurther appreciated that this mode of operation may function whether thecallback integration engine 201 is operating at the firmware 320 or OS220 level, as many mobile operating systems such as ANDROID™ and IOS™allow for applications to integrate with communication features such asphone dialers and incoming call notification prompts.

FIG. 9 is a method diagram illustrating an exemplary callback workflowonce a user selects a callback for an incoming call, according to anembodiment. Initially, a callback button is selected 901 by the user,either from an incoming call that they wish to defer to an automatedcallback or from an email for which they wish to automatically setup acallback, or any other interaction, application, or location where anintegrated callback button may be present. A callback object isinstantiated 902, using a callback integration engine 201, which is anobject with data fields representing the various parts of callback datafor the user and any other callback participants (for example, thecaller if a callback button was pressed on an incoming callnotification, or other individuals participating in an email thread ifthe callback button was pressed from within an email message), and anyrelated information such as what scheduled times may be possible forsuch a callback to take place. This callback object is then stored andmaintained by the callback integration engine 903, updating informationwhen necessary such as to accommodate changes in scheduling or aparticipant indicating that they will be unavailable during the selectedcallback time. This may result in modifying the existing callback toreschedule it, and when the conditions for the callback are met(scheduled time arrives, users are available, or any other conditionsthat may have been set), the callback integration engine 201 initiates acall to each participant 904 and then bridges them into a single call905 where they may interact. This provides automated connection ofmultiple individuals as needed, without requiring any participant toinitiate the call or remember scheduling information, as the entireprocess is handled “behind the scenes” by the callback integrationengine 201.

FIG. 10 is a method diagram illustrating an exemplary callbackprioritization workflow once a data or voice message is received,according to an embodiment. Initially, a data message or voice message(voicemail) may be received 1001 by a computing device 101 wherein theuser of the device is unable to respond to the received messages. Insome embodiments, the voice message may be transcribed into a datamessage 1002 for simpler processing by NLP algorithms. The data messagesare then assigned to a lobby construct 1003 based upon one or morecharacteristics associated with the data message. The lobby constructsare groups of data messages that have similar or correspondingcharacteristics. For each assigned data message in a lobby, the callbackprioritization engine 202 may retrieve and aggregate data 1004 relatedto the assigned data message. Aggregate data may include previoustext-based communications which have been identified to be related tothe assigned data message. Identifying related information may beconducted using the one or more characteristics of the assigned datamessage. The assigned data message and any aggregate data may be inputinto one or more machine learning algorithms 1005, such as, for example,a natural language processing neural network, to analyze the datamessage and its aggregate data 1006. The analyses of the data messagemay provide, determine, derive, or infer context data (e.g., intent,sentiment, subject matter, etc.) which can provide more informationabout the received data messages. The results of the analyses may beused by the callback prioritization engine 202 to compute a priorityscore for the assigned data messages 1007. Using the computed priorityscore, a callback list may be generated 1008 which provides a hierarchalschedule of callback items that may be displayed to a user device forreview and revision if necessary.

FIG. 11 is a block diagram illustrating an exemplary system architecturefor an exemplary mobile device 101 embodiment with an enhanced callbackprioritization engine 1100 operating at the operating system level andconnected to a cloud callback platform 110, according to an embodiment.According to the embodiment, callback cloud 110 may receive requests viaa plurality of communications networks such as a public switchedtelephone network (PSTN) 103 or the Internet 102. These requests maycomprise a variety of communication and interaction types, for exampleincluding (but not limited to) voice calls over a telephone line, videocalls over a network connection, or live text-based chat such as webchat or short message service (SMS) texting via PSTN 103. Suchcommunications networks may be connected to a plurality of mobiledevices 101 such as a user's smartphone or similar mobile device,according to the particular architecture of communication networkinvolved. Mobile devices 101 may be connected to respectivecommunications networks via a variety of means, which may includetelephone dialers, VOIP telecommunications services, web browserapplications, SMS text messaging services, or other telephony or datacommunications services. It will be appreciated by one having ordinaryskill in the art that such means of communication are exemplary, andmany alternative means are possible and becoming possible in the art,any of which may be utilized as an element of system 100 according tothe invention.

When needed (for example, when a user manually requests a callback orwhen configured settings determine that a callback is needed), a user'smobile device 101 creates a session with a cloud callback platform 110with a profile manager 111, resulting in a callback being required.Profile manager 111 receives initial requests to connect to callbackcloud 110, and forwards relevant user profile information to a callbackmanager 113, which may further request environmental context data froman environment analyzer 112. Environmental context data may include (forexample, and not limited to) recorded information about when a user maybe suspected to be driving or commuting from work (as may be determinedfrom device information, such as whether a “do not disturb whiledriving” feature has been enabled, or if the mobile device 101 iscurrently connected to a car audio system), or if the user may be busyfor another reason, such as if they are running or working out (as maybe determined from device sensor data), for example, and may be parsedfrom online profiles or online textual data, using an environmentanalyzer 112.

A callback manager 113 centrally manages all callback data, creating acallback object which may be used to manage the data for a particularcallback, and communicates with an interaction manager 114 which handlesrequests to make calls and bridge calls, which go out to a media server115 which actually makes the calls as requested. In this way, the mediaserver 115 may be altered in the manner in which it makes and bridgescalls when directed, but the callback manager 113 does not need toadjust itself, due to going through an intermediary component, theinteraction manager 114, as an interface between the two. A media server115, when directed, may place calls and send messages, emails, orconnect voice over IP (“VoIP”) calls and video calls, to users over aPSTN 103 or the Internet 102. Callback manager 113 may work with auser's profile as managed by a profile manager 111, with environmentalcontext from an environment analyzer 112 as well as (if provided) EWTinformation for any callback recipients (for example, contact centeragents with the appropriate skills to address the callback requestor'sneeds, or online tech support agents to respond to chat requests), todetermine an appropriate callback time for the two users (a callbackrequestor and a callback recipient), interfacing with an interactionmanager 114 to physically place and bridge the calls with a media server115.

The cloud callback platform 110 may further include an applicationprogramming interface (API) manager 116 and a callback prioritizationengine 117, according to an embodiment. The API manager 116 may be usedto expose or connect with software related to a callback requester and acallback recipient. For example, API manager 116 may be used to connectwith email, messaging, telephone, or other voice communicationapplications present on a mobile device in order to retrieve text-baseddata related to emails, text messages such as SMS and from othermessaging applications, or voice data such as voicemails. Voice datasuch as, for example, a voice mail may be transcribed into a text-baseddata message by the cloud callback platform 110. The retrievedtext-based data messages may be sent to a prioritization engine 117which may identify one or more characteristics of the text-based datamessages. The prioritization engine 117 may use the environment analyzerto parse or analyze the retrieved text-based data messages. Analysis ofthe text-based data messages may be used to determine or infer a varietyof metrics pertaining to a callback requester and at least one callbackrecipient, such as (but not limited to) relationships, intent, subjectmatter, and sentiment. Based on the analysis of the text-based datamessages, the callback prioritization engine 117 may produce priorityscore for each text-based data message which can be used to orderlobbies (callback groups) in order of priority.

According to the embodiment, a mobile device 101 comprises severalhardware and software components operating at various levels to providevarious functions. At the most basic hardware level 210, the physicalhardware of the mobile device 101 may comprise a processor and memory211 that provide standard computing device functionality as described indetail below, with reference to FIGS. 14-17 (grouped together here forthe sake of clarity, it should be appreciated that processor/memory mayor may not be part of the same hardware component, such as asystem-on-a-chip), a baseband processor 212 that manages radio-basedcommunication functions such as cellular or Wi-Fi connectivity, as wellas any of a number of hardware sensors 213 such as (for example,including but not limited to) an accelerometer to detect or measuredevice movement, gyroscope for detecting or measuring deviceorientation, barometer for measuring ambient environment conditions, orglobal positioning system (GPS) receiver for geolocating the device.

An operating system (OS) 220 comprises the main software operating onmobile device 101 and providing various software-based functions such assoftware applications and communications. Various software applications221 operating within (for example in an application layer not shown herebut as is commonly used in various computing devices according to thestandard OSI process model of computing systems) on the mobile device101 may expose and provide access to, or interaction with, varioushardware and sensor features such as to enable a user to view orcalibrate sensor readings.

According to the embodiment, a callback integration engine 201 maycomprise local (that is, operating on the mobile device) functionalitysimilar to a cloud-based callback manager 113 described above, and mayoperate as an application or feature at the operating system level 220,running at a similar privilege level and having similar access tohardware and software functions as other applications 221 operating onmobile device 101. This enables easy installation or removal of callbackintegration engine 201, as it may be readily distributed via similarmethods as any other software application (for example, via anapplication store or similar download portal). However, this mayrestrict the capabilities of callback integration engine 201, as it willhave only indirect access to hardware features (as it will only be ableto access whatever features are exposed by the operating system) and inmany cases may be “sandboxed” and prevented from directly accessingother software or information on the mobile device 201 (for example, ina sandboxed software environment such as APPLE IOS™ or similar).

According to the embodiment, an enhanced callback prioritization engine1100 may comprise local (that is, operating on the mobile device)functionality similar to a cloud-based prioritization engine 117described above, and may operate as an application or feature at theoperating system level 220, running at a similar privilege level andhaving similar access to hardware and software functions as otherapplications 221 operating on mobile device 101. The enhanced callbackprioritization engine 1100 may be included within the same applicationas the callback integration engine 201 or it may be packaged as its ownapplication with built in functionality to communicate with and sharedata with a separate callback integration engine 201. This enables easyinstallation or removal of an enhanced callback prioritization engine1100, as it may be readily distributed via similar methods as any othersoftware application (for example, via an application store or similardownload portal).

The callback integration engine 201 may intercept (e.g., receive, hook,access, or otherwise obtain) voice messages (i.e., voicemail) andtext-based data messages such as (but not limited to) short messageservice (SMS), emails, and group chats sent to the mobile device 101 androute them to an enhanced callback prioritization engine 1100 as acallback object associated with the intercepted data or voice messages.In an embodiment, the mobile device 101 may transcribe voice messagesinto text-based data message for further processing. Each text-baseddata message and voice message (or its transcription) may have one ormore characteristics, such as indications of source, destination, datamessage subject matter, end user identifiers or account identifiers.Application 221 data associated with the data message is aggregated andlinked to the callback object. For example, a first email sent to amobile device 101 may be intercepted and then the enhanced callbackprioritization engine 1100 can identify and retrieve other emails, textmessages, group chats, financial transactions, social media interactionsfrom application 221 on the phone 101 and associated with the firstemail in order to provide more context for the first email message. Inthis example, other communications exchanged between the first emailrecipient and the first email sender (such as an email thread, textmessages, or voicemails) may be aggregated and added to the callbackobject associated with the first email. The aggregated data may then befed into one or more machine learning algorithms that analyze theaggregated data and can generate for each data message a priority score.The priority score may be used to create a hierarchy or priority list ofcallback objects such that each callback may be scheduled and executedin order of highest priority to lowest priority.

According to another embodiment, an exemplary mobile device 101 with acallback integration engine 201 and an enhanced callback prioritizationengine 1100 can operate at the device firmware level, according to anembodiment. According to the embodiment, a firmware level encompasseslow-level program code that operates “below” the operating system 220,operating directly on hardware components of mobile 101, and comprisessuch features as a baseband controller that controls features ofbaseband processor 212 and the operation of which is fully transparentto the user (that is, a user of mobile device 101 does not see orinteract with firmware, and many users may not even be aware of itsexistence or capabilities).

According to the embodiment, a callback integration engine 201 and anenhanced callback prioritization engine 1100 may operate as a firmwarecomponent that directly interfaces with hardware features of mobiledevice 101, enabling direct control as needed while exposing desiredfunctionality to the operating system 220 in a transparent manner (thatis, the operating system may only know that a feature is available, andmay not be able to determine that the feature is provided by thecallback integration engine 201 or callback prioritization engine 202rather than, for example, the baseband controller). This enables moredirect control over device functions, such as call and data messagerouting and hardware information such as sensor readings (as may be usedin callback workflow processing), and allows the callback integrationengine 201 to intercept incoming calls and radio information (e.g., datamessages) before it reaches the operating system, so data may bemanipulated and additional features may be integrated prior topresentation to the OS. This also prevents accidental removal ofcallback integration engine 201 and an enhanced callback prioritizationengine 1100, as a user may have no control over device firmware andanything operating at or below the OS level 220 would inherently haveinsufficient access to modify firmware. This further enables callbackintegration engine 201 and an enhanced callback prioritization engine1100 to access hardware features that may be restricted or completelyprevented when operating within the OS layer 220, for example hardwaresensor or geolocation information that may be incorporated into acallback workflow.

According to another embodiment, wherein the enhanced callbackprioritization engine 1100 runs at the operating system 220 level, anagent 1101 may be included with the enhanced callback prioritizationengine 1100 that is provided with root access and privileges in order tomonitor the mobile device and access all files and folders needed toachieve to retrieval of third-party app data for use in the machinelearning analysis.

FIG. 12 is a block diagram illustrating an exemplary system architecturefor an enhanced callback prioritization engine 1100, according to anembodiment. According to the embodiment, an enhanced callbackprioritization engine 1100 may receive a callback object with a datamessage comprising a characteristic from a callback integration engine201. The data message can have part of sessions that includeasynchronous text, voice message, or data message (e.g., not a livevoice conversation such as a telephone call) communications between andend user and a computing device (e.g., mobile device, contact centeragent [e.g., a person, bot, or combination thereof], etc.). In someexamples, an enhanced callback prioritization engine 1100 can receivevoice messages that have been send as audio messages (voicemail) orconverted into text messages. The sessions can occur continuously in aconversational manner, or can be intermittent, e.g., lasting over hours,days, or weeks with dormant periods between communications. The enhancedcallback prioritization engine 1100 receiving data messages withcharacteristics can then find other relevant application data using theinformation in the callback object.

The enhanced callback prioritization engine 1100 may identify andaggregate other data related to the data message. For example, if anassigned data message is part of an ongoing session (such as an emailthread, text, or group messaging conversation) then a data aggregator402 may retrieve the other messages contained within the session andappend or link the other messages to the assigned data message.Additionally, if the assigned data message is a standalone message(i.e., sent with no connection to a prior session) then the enhancedcallback prioritization engine 1100 may scan for and identify othercommunications that may be related to the assigned data message. Othercommunications may be identified as related based upon one or morecharacteristics of the assigned data message. For example, an end usermay receive an email asking for a response of some sort from a coworkerwhich may be related to previous communications exchanged between theend user and the coworker, but not contained within the same thread asthe previous communications. In this example, the enhanced callbackprioritization engine 1100 may be able to identify the previouscommunications as related to the original email asking for a responseand append or link them to the assigned data messages generated from theresponse email. As another example, an end user mobile device 101 mayreceive a voicemail from an individual or entity (business) asking theend user to callback when available. The voicemail may be transcribed toa text data message and have one or more characteristics, such as thephone number or other identifier associated with the voicemail, and thedata aggregator 1201 can use the voicemail characteristics to identifyan email (or other data message) that may be associated with thevoicemail. For example, an employee of a company may send to a secondemployee a voicemail requesting a callback and further, the twoemployees may have been emailing or text messaging each other earlier inthe day or week. These emails or text messages may be linked to thetranscribed voicemail by the data aggregator 1201. In this way the dataaggregator 1201 may identify and aggregate application data correlatingwith a data message.

Social media apps may also be associated with a callback object. Forexample, two competing callback requests, both scoring the same in acallback queue—may be resolved by determining recent social mediainteractions. If social media interaction is higher between one of thecallback requesters and the user, then that callback request can beplaced in a higher priority than the other. Social media history maycomprise likes, mentions, thumbs-up/down, or other means the serviceuses to indicate sentiment. If the caller is a moderator oradministrator on a forum or other outlet where the user is a member orhas recent activity, then higher priority can be given over regular ornon-members.

Financial apps may also be used to determine if a callback requester isassociated with an invoice, pay request, or other financial transaction.Financial app integration may inform the machine learning algorithms1202 to prioritize callbacks based on transactions amounts, e.g., afirst callback requester who sent the user a large monetary sum is givenpriority over a second callback requester that has sent a small amountof funds.

The assigned data message and any linked and aggregated data may then besent as input to a machine learning (ML) engine 1202 for processing andanalysis. The ML engine 1202 may comprise one or more machine learningalgorithms such as, for example, a natural language processing (NLP)neural network. The NLP may be used for text classification functionsincluding, but not limited to, sentiment analysis, topic categorization,language recognition, and intent analysis. The NLP may be used toperform sentiment analysis on the assigned data message and anyavailable aggregated data linked to the data message. Sentiment analysismay be used to determine if the content of a data message (and itsaggregate data) is positive, negative, or neutral. This could be used toclassify data messages to better respond to inquiries or requests. TheNLP may be used to determine urgency. Identifying keywords andsentiments that provide insight and context into the callback request.The NLP may be used for topic categorization to determine which categorya data message belongs to, essentially this constitutes a robust,automatic method of tagging and organizing the content of a data messageor its aggregate data. The NLP may be used to understand the intent of adata message or its aggregate data by determining the underlyingintention of a given data message. In addition, the NLP may be used tofind and identify relationships between a caller and callee, or amessage recipient and a message sender, or between and among a group ofpeople (i.e., multiple people in an email thread or group text message).Using the identified relationships, an enhanced callback prioritizationengine 1100 or callback integration engine 201 may generate personalizedsmart replies based on the identified relationships and otherinformation (intent, sentiment, urgency, topic) derived from the NLP.The relationships may be used to determine the priority of an assigneddata message. Additionally, NLP can also be used to determine urgency ofa callback. Given certain keywords such as “it's important”, “rightaway”, and “urgent” can be used to give some callback requests higherpriority than others.

In an embodiment, the ML engine 1202 may also comprise one or more audiomachine learning algorithms which may be used for audio classificationand speech recognition functions. Audio classification ML algorithms maybe used when a voice message (voicemail) is received by mobile device101 or by an enhanced callback prioritization engine 1100. The audioalgorithm may be used to identify a speaker by taking a snippet of thevoice message to create a unique identifier, like a voice fingerprint.Then the fingerprint may be input into a neural network that can matchthe unique identifier to a known or previously heard voice. This mayallow an enhanced callback prioritization engine 1100 to identifycallers who may be calling from a new device or using a new identifierthat may not be known by the callback integration engine 201.Furthermore, an audio ML algorithm may be utilized for speechrecognition functions such as voice to text technology (i.e.,transcribing a voicemail into a text data message) or to provide voiceassistant interaction (e.g., in a contact center environment whereby acustomer may communicate with a bot or script voice assistant).

There are various benefits of having a ML engine 1202 operating locallyon a mobile device compared to operating in a cloud environment whichthe mobile device may connect to. Operating the ML engine 1202 on amobile device 101 protects against low latency. As most ML algorithmsmay be operating in real-time, network speed may be too slow to transmita live data feed from the device to the cloud callback platform 110, andthen back to the device. Furthermore, running the ML engine 1202 locallyon a mobile device 101 removes the risk of low or no network connectionshalting the ML processes. In a mobile device 101 the ML engine 1202 maystill work when run on device, even if low or no connectivity. Operatingthe ML engine 1202 locally on a mobile device 101 facilitates privacyover the mobile device user's data. ML needs data to build a usefulmodel and to make useful inferences derived from the model, however,some data may be sensitive (e.g., personal identifying information,regulated data). The enhanced callback prioritization engine 1100 canuse and secure data on the device, without having to transmit anysensitive data.

The ML engine 1202 may forward the results of the analyses (e.g.,inferences, intent, sentiment, urgency, topic, etc.) of the various MLalgorithms to a prioritization module 1203. The prioritization module1202 may then use the results of the analyses (e.g., inferences, intent,sentiment, urgency, topic, etc.) to generate a priority score for eachdata message. For example, if multiple assigned messages within aconstruct have been identified to belonging to the same topic, then themost recent data message pertaining to the shared topic may be given ahigher priority score and thus the callback associated with that datamessage would be executed before the previously received data messages.For example, a cable company service agent may have twenty five datamessages, and of those data messages twenty are about a loss of cableservice. In this example, the ML engine 1202 may be able to identifythat each of those twenty data messages originated from the same areacode and therefore there must be some service disruption affecting thatarea code. The prioritization module 1203 can use this information tomove the first of the twenty people up in priority so that the problemwith their cable service may be addressed. If the cable service agent isable to resolve the issue for the first of twenty data messages relatingto cable service loss, then the enhanced callback prioritization engine1100 may be able to send an automated message to the other of thecustomers associated with the remaining nineteen data messages providinginformation about the loss of cable service. For example, an automatedmessage may be sent which says, “Is your call about loss of cableservice? If so, we are aware, and your service should be restored soon.Would you like to request a callback?”. If the receiver of the automatedmessage replies with a callback request, then the data message mayremain in the queue, otherwise the data message will be removed from thecallback queue.

Intent and sentiment analysis results may be used to determine priorityas well. For example, the sentiment of one data message may bedetermined to be positive, whereas the sentiment of another data messagemay be urgent, or negative. A data message with an urgent or negativesentiment may be given a higher priority score than a data message witha positive sentiment in order to account for the urgency of the datamessage, or to quickly respond to the negative sentiment data message inorder to change sentiment of the data message sender to neutral orpositive. Additionally, the intent of a data message may be used todetermine the priority of response. For example, a data message mayindicate that there was no intent for a callback or response, whereasanother data message may have an inferred intent for a callback. In thiscase, the data message with the inferred intent for a callback will begiven a higher priority score than the data message with little to noinferred callback intent.

Based upon the priority score of each data message, the prioritizationmodule 404 may reorder the callback queue in a hierarchal manner wherethe highest priority data message is at the top of the queue and theother data messages are sorted in descending order of priority. Theprioritization module 1203 may also produce a callback list based uponthe sorted queue. The callback list may include the original datamessage, any aggregate data, and data contained within the callbackobject associated with each data message. For example, the callback listmay display the original data message, a scheduled callback time, andthe name of the callback recipient. The callback list may be sent to themobile device 101 (or any computing device) for display and interactionby the mobile device user.

An additional anticipated feature of the instant invention isincentivized on-device callback management. This is where rewards areoffered to the mobile device user for handling a callback in a specifiedmanner. This could be a specific time—e.g., “call back within thiswindow for a bonus!”, or “if you call back, use code ABC for a discount;only valid for twenty-four hours” Specific channels could beincentivized such as asking a user to reach out on a specific platform.Rewards could be in the form of in-game rewards, money, discounts,add-ons to their mobile service plan, priority handling on their nextinitiated interaction, or discounted or free service provided by thecaller—e.g., “get in touch with us via our streaming channel for a freemonth!”.

Incentivized on-device callback management may be implemented by usingNLP to determine the incentives from the data message and adding afunction to each callback object meeting the criteria of the datamessage's incentive. For example, if a user is given an incentive tocall back within 24 hours to receive 20% off of becoming a new member ofa service, a message or notification may be generated on the usersdevice alerting them of the offer and providing a link to the specificcallback object on the callback list.

FIG. 13 is a method diagram illustrating several app integrations forenhancing callback prioritization. This diagram illustrates features ofthe data aggregator 1201 using APIs and other functions (e.g., screenmonitoring, administrative access, etc.) to retrieve information fromthe mobile device in order to provide context to callback requestscoring and ordering. When a characteristic is known about the callbackrequest 1301—e.g., caller name, location, business, demographics,subject matter, etc., the data aggregator 1201 retrieves, from theapplications in the mobile device, any communications 1302 from the samecaller, business, subject matter, etc. If enough data points from themachine learning 1202 suggest a communication or interaction from anexternal application is relevant, that communication or interaction maybe used with the original callback request analysis to provide somecontext with regards to sentiment, urgency, relationships, and intent.The higher the machine learning confidence is for the sentiment,urgency, relationships, and intent of the external application data inrelation to the callback request, the higher the value of aprioritization score will be 1302. This same analysis and scoringcontinue for social media interactions 1303/1303 a, financialinteractions 1304/1304 a, and gaming interactions 1305/1305 a. Sensorsand sensor history may be used, apart from or alongside of calendar orto-do apps to determine if the caller and the user have been in the samearea or met physically recently 1306/1306 a. For example, after a recenttrip to a specific business in China, an American business executive mayreceive a follow-up call from the business clerk. The callback willreach a high priority based on the travel data and nature of the user'sbusiness.

Example sentiments, relationships, and intents comprise, but are notlimited to scheduled events involving the caller and recent eventsinvolving caller determined by GPS history or shared events orcalendars. Whether a caller is a team member, party member, or acompetitor. Whether the caller a developer, moderator, player, orotherwise involved in a game the user plays. Recent interactions onsocial media may be analyzed such as likes, mentions, thumbs-up/down, orsome means the service uses to indicate sentiment. The caller may amoderator or administrator on a forum or other outlet where the user isa member or has recent activity. It could be determined if a caller owesthe user money, the user owes the caller money, or the caller and userhave exchanged money recently.

Each prioritization score 1302 a, 1303 a, 1304 a, 1305 a, 1306 a is thensummed 1307 and used to place the callback object in the appropriateplace in the callback list 1308. Prioritization scores 1302 a, 1303 a,1304 a, 1305 a, 1306 a may fall on any scale or range desired and may benormalized as seen fit, evident to those with ordinary skill in the art.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (“ASIC”), or on a network interface card.

Software/hardware hybrid implementations of at least some of the aspectsdisclosed herein may be implemented on a programmable network-residentmachine (which should be understood to include intermittently connectednetwork-aware machines) selectively activated or reconfigured by acomputer program stored in memory. Such network devices may havemultiple network interfaces that may be configured or designed toutilize different types of network communication protocols. A generalarchitecture for some of these machines may be described herein in orderto illustrate one or more exemplary means by which a given unit offunctionality may be implemented. According to specific aspects, atleast some of the features or functionalities of the various aspectsdisclosed herein may be implemented on one or more general-purposecomputers associated with one or more networks, such as for example anend-user computer system, a client computer, a network server or otherserver system, a mobile computing device (e.g., tablet computing device,mobile phone, smartphone, laptop, or other appropriate computingdevice), a consumer electronic device, a music player, or any othersuitable electronic device, router, switch, or other suitable device, orany combination thereof. In at least some aspects, at least some of thefeatures or functionalities of the various aspects disclosed herein maybe implemented in one or more virtualized computing environments (e.g.,network computing clouds, virtual machines hosted on one or morephysical computing machines, or other appropriate virtual environments).

Referring now to FIG. 14 , there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one embodiment, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one embodiment, a computing device 10 may beconfigured or designed to function as a server system utilizing CPU 12,local memory 11 and/or remote memory 16, and interface(s) 15. In atleast one embodiment, CPU 12 may be caused to perform one or more of thedifferent types of functions and/or operations under the control ofsoftware modules or components, which for example, may include anoperating system and any appropriate applications software, drivers, andthe like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some embodiments, processors 13 may includespecially designed hardware such as application-specific integratedcircuits (ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a specific embodiment,a local memory 11 (such as non-volatile random access memory (RAM)and/or read-only memory (ROM), including for example one or more levelsof cached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one embodiment, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (Wi-Fi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity A/V hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 14 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe inventions described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one embodiment, a single processor 13 handles communicationsas well as routing computations, while in other embodiments a separatededicated communications processor may be provided. In variousembodiments, different types of features or functionalities may beimplemented in a system according to the invention that includes aclient device (such as a tablet device or smartphone running clientsoftware) and server systems (such as a server system described in moredetail below).

Regardless of network device configuration, the system of the presentinvention may employ one or more memories or memory modules (such as,for example, remote memory block 16 and local memory 11) configured tostore data, program instructions for the general-purpose networkoperations, or other information relating to the functionality of theembodiments described herein (or any combinations of the above). Programinstructions may control execution of or comprise an operating systemand/or one or more applications, for example. Memory 16 or memories 11,16 may also be configured to store data structures, configuration data,encryption data, historical system operations information, or any otherspecific or generic non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device embodiments may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine-readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some embodiments, systems according to the present invention may beimplemented on a standalone computing system. Referring now to FIG. 15 ,there is shown a block diagram depicting a typical exemplaryarchitecture of one or more embodiments or components thereof on astandalone computing system. Computing device 20 includes processors 21that may run software that carry out one or more functions orapplications of embodiments of the invention, such as for example aclient application 24. Processors 21 may carry out computinginstructions under control of an operating system 22 such as, forexample, a version of MICROSOFT WINDOWS™ operating system, APPLE OSX™ oriOS™ operating systems, some variety of the Linux operating system,ANDROID™ operating system, or the like. In many cases, one or moreshared services 23 may be operable in system 20, and may be useful forproviding common services to client applications 24. Services 23 may forexample be WINDOWS™ services, user-space common services in a Linuxenvironment, or any other type of common service architecture used withoperating system 21. Input devices 28 may be of any type suitable forreceiving user input, including for example a keyboard, touchscreen,microphone (for example, for voice input), mouse, touchpad, trackball,or any combination thereof. Output devices 27 may be of any typesuitable for providing output to one or more users, whether remote orlocal to system 20, and may include for example one or more screens forvisual output, speakers, printers, or any combination thereof. Memory 25may be random-access memory having any structure and architecture knownin the art, for use by processors 21, for example to run software.Storage devices 26 may be any magnetic, optical, mechanical, memristor,or electrical storage device for storage of data in digital form (suchas those described above, referring to FIG. 14 ). Examples of storagedevices 26 include flash memory, magnetic hard drive, CD-ROM, and/or thelike.

In some embodiments, systems of the present invention may be implementedon a distributed computing network, such as one having any number ofclients and/or servers. Referring now to FIG. 16 , there is shown ablock diagram depicting an exemplary architecture 30 for implementing atleast a portion of a system according to an embodiment of the inventionon a distributed computing network. According to the embodiment, anynumber of clients 33 may be provided. Each client 33 may run softwarefor implementing client-side portions of the present invention; clientsmay comprise a system 20 such as that illustrated in FIG. 15 . Inaddition, any number of servers 32 may be provided for handling requestsreceived from one or more clients 33. Clients 33 and servers 32 maycommunicate with one another via one or more electronic networks 31,which may be in various embodiments any of the Internet, a wide areanetwork, a mobile telephony network (such as CDMA or GSM cellularnetworks), a wireless network (such as Wi-Fi, WiMAX, LTE, and so forth),or a local area network (or indeed any network topology known in theart; the invention does not prefer any one network topology over anyother). Networks 31 may be implemented using any known networkprotocols, including for example wired and/or wireless protocols.

In addition, in some embodiments, servers 32 may call external services37 when needed to obtain additional information, or to refer toadditional data concerning a particular call. Communications withexternal services 37 may take place, for example, via one or morenetworks 31. In various embodiments, external services 37 may compriseweb-enabled services or functionality related to or installed on thehardware device itself. For example, in an embodiment where clientapplications 24 are implemented on a smartphone or other electronicdevice, client applications 24 may obtain information stored in a serversystem 32 in the cloud or on an external service 37 deployed on one ormore of a particular enterprise's or user's premises.

In some embodiments of the invention, clients 33 or servers 32 (or both)may make use of one or more specialized services or appliances that maybe deployed locally or remotely across one or more networks 31. Forexample, one or more databases 34 may be used or referred to by one ormore embodiments of the invention. It should be understood by one havingordinary skill in the art that databases 34 may be arranged in a widevariety of architectures and using a wide variety of data access andmanipulation means. For example, in various embodiments one or moredatabases 34 may comprise a relational database system using astructured query language (SQL), while others may comprise analternative data storage technology such as those referred to in the artas “NoSQL” (for example, HADOOP CASSANDRA™, GOOGLE BIGTABLE™, and soforth). In some embodiments, variant database architectures such ascolumn-oriented databases, in-memory databases, clustered databases,distributed databases, or even flat file data repositories may be usedaccording to the invention. It will be appreciated by one havingordinary skill in the art that any combination of known or futuredatabase technologies may be used as appropriate, unless a specificdatabase technology or a specific arrangement of components is specifiedfor a particular embodiment herein. Moreover, it should be appreciatedthat the term “database” as used herein may refer to a physical databasemachine, a cluster of machines acting as a single database system, or alogical database within an overall database management system. Unless aspecific meaning is specified for a given use of the term “database”, itshould be construed to mean any of these senses of the word, all ofwhich are understood as a plain meaning of the term “database” by thosehaving ordinary skill in the art.

Similarly, most embodiments of the invention may make use of one or moresecurity systems 36 and configuration systems 35. Security andconfiguration management are common information technology (IT) and webfunctions, and some amount of each are generally associated with any ITor web systems. It should be understood by one having ordinary skill inthe art that any configuration or security subsystems known in the artnow or in the future may be used in conjunction with embodiments of theinvention without limitation, unless a specific security 36 orconfiguration system 35 or approach is specifically required by thedescription of any specific embodiment.

FIG. 17 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to peripherals such as a keyboard49, pointing device 50, hard disk 52, real-time clock 51, a camera 57,and other peripheral devices. NIC 53 connects to network 54, which maybe the Internet or a local network, which local network may or may nothave connections to the Internet. The system may be connected to othercomputing devices through the network via a router 55, wireless localarea network 56, or any other network connection. Also shown as part ofsystem 40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various embodiments, functionality for implementing systems ormethods of the present invention may be distributed among any number ofclient and/or server components. For example, various software modulesmay be implemented for performing various functions in connection withthe present invention, and such modules may be variously implemented torun on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various embodiments described above. Accordingly, the presentinvention is defined by the claims and their equivalents.

What is claimed is:
 1. A mobile device with active callbackprioritization, comprising: a processor, a memory, and a plurality ofprogramming instructions stored in the memory and operable on theprocessor; a callback integration engine comprising a subset of theplurality of programming instructions that, when operating on theprocessor, cause the processor to: receive a data or voice message, thedata or voice message comprising at least one characteristic; produce acallback object in memory comprising information associated with thedata or voice message received; and send the callback object to anenhanced callback prioritization engine; and the enhanced callbackprioritization engine comprising a subset of the plurality ofprogramming instructions that, when operating on the processor, causethe processor to: receive the callback object from the callbackintegration engine; retrieve and aggregate application data related tothe data or voice message; use the callback object and the aggregatedapplication data as inputs into one or more machine learning algorithms,wherein the machine learning algorithms analyze the callback object'sinformation and the aggregated application data to determine the contextand urgency associated with the data or voice message; for each callbackobject, compute a priority score based at least upon the results of theanalysis; and use the computed priority score, the callback object data,and the data or voice message to generate a callback list.
 2. The mobiledevice of claim 1, wherein the application data is retrieved usingapplication programming interfaces.
 3. The mobile device of claim 1,wherein the callback list comprises a smart reply message.
 4. The mobiledevice of claim 1, wherein the callback integration engine receives auser confirmed callback list and executes the confirmed callback itemson the list.
 5. The mobile device of claim 1, wherein the applicationdata comprises data from communication, social media, financial, gaming,and productivity applications.
 6. The mobile device of claim 1, whereinthe machine learning algorithms comprise natural language processing. 7.A method for active callback prioritization, comprising the steps of:receiving a data or voice message, the data or voice message comprisingat least one characteristic; producing a callback object in memorycomprising information associated with the data or voice messagereceived; retrieving and aggregating application data related to thedata or voice message; using the callback object and the aggregatedapplication data as inputs into one or more machine learning algorithms,wherein the machine learning algorithms analyze the callback object'sinformation and the aggregated application data to determine the contextand urgency associated with the data or voice message; for each callbackobject, computing a priority score based at least upon the results ofthe analysis; and using the computed priority score, the callback objectdata, and the data or voice message to generate a callback list.
 8. Themethod of claim 7, wherein the application data is retrieved usingapplication programming interfaces.
 9. The method of claim 7, whereinthe callback list comprises a smart reply message.
 10. The method ofclaim 7, wherein a user confirmed callback list is generated andexecutes the confirmed callback items on the list.
 11. The method ofclaim 7, wherein the application data comprises data from communication,social media, financial, gaming, and productivity applications.
 12. Themethod of claim 7, wherein the machine learning algorithms comprisenatural language processing.